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Autonomous  Driving  in  Traffic: 
Boss  and  the  Urban  Challenge 


Chris  Urmson,  Chris  Baker,  John  Dolan,  Paul  Rybski,  Bryan  Salesky, 
William  "Red"  Whittaker,  Dave  Ferguson,  and  Michael  Darms 


■  The  DARPA  Urban  Challenge  was  a  compe¬ 
tition  to  develop  autonomous  vehicles  capable 
of  safely,  reliably,  and  robustly  driving  in  traf¬ 
fic.  In  this  article  we  introduce  Boss,  the 
autonomous  vehicle  that  won  the  challenge. 
Boss  is  a  complex  artificially  intelligent  soft¬ 
ware  system  embodied  in  a  2007  Chevy  Tahoe. 
To  navigate  safely,  the  vehicle  builds  a  model  of 
the  world  around  it  in  real  time.  This  model  is 
used  to  generate  safe  routes  and  motion  plans 
both  on  roads  and  in  unstructured  zones.  An 
essential  part  of  Boss's  success  stems  from  its 
ability  to  safely  handle  both  abnormal  situa¬ 
tions  and  system  glitches. 


in  2003  the  Defense  Advanced  Research  Projects  Agency 
(DARPA)  announced  the  first  Grand  Challenge  with  the  goal  of 
developing  vehicles  capable  of  autonomously  navigating  desert 
trails  and  roads  at  high  speeds.  The  competition  was  generated 
as  a  response  to  a  mandate  from  the  United  States  Congress  that 
a  third  of  U.S.  military  ground  vehicles  he  unmanned  hy  2015. 
To  achieve  this  goal  DARPA  drew  from  inspirational  successes 
such  as  Charles  Lindbergh's  May  1927  solo  transatlantic  flight 
aboard  the  Spirit  of  St.  Louis  and  the  May  1997  IBM  Deep  Blue 
chess  victory  over  reigning  world  chess  champion  Garry  Kas¬ 
parov.  Instead  of  conventional  funded  research,  these  successes 
were  organized  as  challenges  with  monetary  prizes  for  achieving 
specific  goals. 

DARPA's  intent  was  not  only  to  develop  technology  but  also 
to  rally  the  field  of  robotics  and  ignite  interest  in  science  and 
engineering  on  a  scale  comparable  to  the  Apollo  program.  Thus, 
the  Grand  Challenges  were  born.  Organized  as  competitions 
with  cash  prizes  awarded  to  the  winners  ($1  million  in  2004;  $2 
million  in  2005;  and  $2  million  for  first,  $1  million  for  second, 
and  $500,000  for  third  in  2007).  For  the  first  two  years  the  gov¬ 
ernment  provided  no  research  funding  to  teams,  forcing  them 
to  be  resourceful  in  funding  their  endeavours.  For  the  2007 
Urban  Challenge,  some  research  funding  was  provided  to  teams 
through  research  contracts.  By  providing  limited  funding,  the 
competition  encouraged  teams  to  find  corporate  sponsorship, 
planting  the  seeds  for  strong  future  relationships  between  aca¬ 
demia  (the  source  of  many  of  the  teams)  and  industry. 

The  Grand  Challenge^  was  framed  as  a  cross-country  race 
where  vehicles  had  to  drive  a  prescribed  route  in  the  minimum 
time.  There  would  be  no  moving  obstacles  on  the  course,  and 
no  one  was  allowed  to  predrive  the  course.  To  complete  the 
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Figure  1.  The  Urban  Challenge  Final  Event. 


Eleven  teams  qualified  for  the  Urban  Challenge  final  event.  Eighty-nine  teams  from  the  United  States  and  countries  as  far  away  as  China 
registered  for  the  competition. 


challenge,  the  autonomous  vehicles  would  need  to 
drive  the  route  in  less  than  10  hours.  The  compe¬ 
tition  was  scoped  to  require  driving  both  on  and 
off  road,  fhrough  underpasses,  and  even  fhrough 
standing  water.  These  diverse  challenges  lead  to  a 
field  wifh  a  wide  variefy  of  vehicles,  ranging  from 
milifary  high-mobilify  mulfipurpose  wheeled 
vehicles  (HMMWVs)  and  sport  utility  vehicles 
(SUVs)  to  motorcycles  and  custom-built  dune  bug¬ 
gies. 

The  first  running  of  the  Grand  Challenge 
occurred  on  March  3,  2004.  The  142-mile  course 
was  too  difficult  for  the  field,  with  the  best  vehicle 
traveling  7.4  miles.  Though  described  as  a  failure 
in  the  popular  press,  the  competition  was  a  tech¬ 
nical  success;  though  the  vehicles  drove  a  small 
fraction  of  fhe  course,  the  top  competitors  did  so  at 
high,  sustained  speeds  that  had  not  been  previ¬ 
ously  demonstrated.  Given  the  excitement  and 
promise  generated  by  this  first  challenge,  the  com¬ 
petition  was  rerun  in  2005. 

The  2005  Grand  Challenge  began  at  6:30  am  on 
October  8.  Vehicles  launched  at  intervals  from 
fhree  start  chutes  in  the  Mojave  desert.  Approxi¬ 
mately  7  hours  later,  the  first  of  five  finishers  com- 
plefed  fhe  challenge.  Stanley  (Thrun  et  al.  2006), 
from  Sfanford  Universify,  finished  first,  in  6  hours 
and  53  minutes,  followed  soon  affer  by  the  two 
Carnegie  Mellon  robots.  Sandstorm  and 
Highlander  (Urmson  et  al.  2006).  Kat-5,  funded  by 
fhe  Gray  Insurance  Company,  and  TerraMax,  a 
30,000-pound  behemoth  developed  by  the 
OshKosh  Truck  Corporation,  rounded  out  the  fin¬ 
ishers.  With  five  vehicles  completing  the  course, 
the  2005  Grand  Challenge  redefined  the  percep¬ 
tion  of  fhe  capabilify  of  aufonomous  mobile 
robofs. 

The  DARPA  2007  Urban  Challenge^  continued 
the  excitement  generated  by  the  previous  chal¬ 
lenges,  engaging  researchers  from  around  fhe 


world  to  advance  intelligent,  autonomous  vehicle 
technology  (figure  1).  Competitors  were  required 
to  develop  full-size  aufonomous  vehicles  cable  of 
driving  with  moving  traffic  in  an  urban  environ¬ 
ment  at  speeds  up  to  30  miles  per  hour  (mph).  The 
vehicles  were  required  to  demonstrate  capabilities 
that  were  undeniably  intelligent,  including  obey¬ 
ing  traffic  rules  while  driving  along  roads,  negofi- 
ating  intersections,  merging  with  moving  traffic, 
parking,  and  independently  handling  abnormal 
situations. 

The  Urban  Challenge  was  fundamentally  about 
intelligent  software.  The  most  visible  artifacts  of 
fhe  research  and  development  are  the  numerous 
autonomous  vehicles  that  were  entered  in  the 
competition,  but  the  magic  is  really  in  the  algo¬ 
rithms  that  make  these  vehicles  drive.  The  compe¬ 
tition  was  a  60-mile  race  through  urban  traffic;  a 
drive  fhaf  many  people  face  daily.  Thus  fhe  chal¬ 
lenge  was  not  in  developing  a  mechanism  that 
could  survive  these  rigors  (most  automotive  man¬ 
ufacturers  do  this  already),  but  in  developing  the 
algorithms  and  software  that  would  robustly  han¬ 
dle  driving,  even  during  unexpected  situations.  To 
complete  the  challenge,  vehicles  had  to  consis¬ 
tently  display  safe  and  correcf  driving  behavior; 
any  unsafe  or  illegal  maneuver  could  resulf  in  elim¬ 
ination  from  the  competition. 

In  this  article  we  present  an  overview  of  Boss, 
fhe  vehicle  fhat  won  the  Urban  Challenge  (figure 
2).  Boss  was  developed  by  a  team  of  researchers 
and  students  from  Carnegie  Mellon  University, 
General  Motors,  Caterpillar,  Continental,  and 
Intel.  Boss  drives  at  speeds  up  to  30  mph  by  mon¬ 
itoring  its  location  and  environment  and  con¬ 
stantly  updating  its  motion  plan  and  the  model  of 
fhe  world  around  if.  Throughout  this  article  we 
provide  pointers  to  other  publications  that  delve 
into  each  area  in  greater  depth. 
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Figure  2.  Boss. 

Boss  uses  a  combination  of  1 7  sensors  to  navigate  safely  in  traffic  at  speeds  up  to  30  mph. 


The  Vehicle 

Boss  is  a  2007  Chevrolet  Tahoe  modified  for 
autonomous  driving.  While  it  was  built  from  the 
chassis  up  for  the  Urban  Challenge,  it  incorporat¬ 
ed  many  lessons  learned  from  the  previous  chal¬ 
lenges.  Like  most  of  the  vehicles  that  competed  in 
the  Urban,  we  built  Boss  from  a  standard  chassis, 
to  reduce  mechanical  complexity  and  benefit  from 
the  reliability  of  a  highly  engineered  platform.  The 
Tahoe  provides  ample  room  to  mount  sensors, 
computers,  and  other  components  while  main¬ 
taining  seating  for  four,  which  is  important  for  safe 
and  efficient  testing  and  development.  Computer 
control  of  the  vehicle  is  made  possible  by  using 
electric  motors  to  actuate  the  brake  pedal,  gear 
selector,  and  steering  column  and  through  con¬ 
troller  area  network  (CAN)  bus  communication 
with  the  engine  control  module.  Despite  these 
changes,  the  Tahoe  maintains  its  standard  human 


driving  controls,  and  pressing  a  button  reverts  it  to 
a  human-driven  vehicle. 

Boss  integrates  a  combination  of  17  different 
sensors  (a  mixture  of  light  detection  and  ranging 
[lidar],  radio  detection  and  ranging  [radar],  and 
global  positioning  system  [GPS]).  These  sensors  are 
mounted  to  provide  a  full  360  degree  field  of  view 
around  the  vehicle.  In  many  cases,  sensor  field  of 
views  overlap,  providing  redundancy  and  in  some 
cases  complemenfary  information.  The  primary 
sensor  is  a  Velodyne  HDL-64  lidar,  mounted  on  the 
top  of  the  vehicle.  This  lidar  provides  dense  range 
data  around  the  vehicle.  Three  Sick  laser  range 
finders  are  used  to  provide  a  virtual  bumper 
around  the  vehicle  while  two  other  long-range 
lidars  are  used  to  track  vehicles  at  ranges  of  up  fo 
100  m.  Three  Confinental  ACC  radars  are  mount¬ 
ed  in  the  front  (two)  and  rear  (one)  bumpers  to 
detect  and  track  moving  vehicles  at  long  ranges.  In 
addition,  a  pair  of  Confinental  sensors  (lidar  and 
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Sensor  Data  Abstraction 


Figure  3.  The  Boss  Software  Architecture. 

The  software  architecture  consists  of  a  perception  layer  that  provides  a  model  of  the  world  to  a  classical  three-tier  con¬ 
trol  architecture. 


radar)  are  mounted  on  pointable  bases,  enabling 
Boss  to  "turn  its  head”  to  look  long  distances  down 
cross  streets  and  merging  roads.  To  determine  its 
location,  Boss  uses  a  combination  of  GPS  and  iner¬ 
tial  data  fused  by  an  Applanix  POS-LV  system.  In 
addition  to  this  system.  Boss  combines  data  from  a 
pair  of  down  looking  SICK  laser  range  finders  to 
localize  itself  relative  to  lane  markings  described  in 
its  on-board  map.  The  number  of  sensors  incorpo¬ 
rated  into  Boss  provides  a  great  deal  of  information 
to  the  perception  system. 

For  computation.  Boss  uses  a  CompactPCl  chas¬ 
sis  with  ten  2.16  gigahertz  Core2Duo  processors, 
each  with  2  gigabytes  of  memory  and  a  pair  of 
gigabit  Ethernet  ports.  Each  computer  boots  off  of 
a  4  gigabyte  flash  drive,  reducing  the  likelihood  of 
a  disk  failure.  Two  of  the  machines  also  mount  500 
gigabyte  hard  drives  for  data  logging,  and  all  are 
time  synchronized  through  a  custom  pulse-per- 
second  adapter  board.  The  computer  cluster  pro¬ 
vides  ample  processing  power,  so  we  were  able  to 
focus  our  efforts  on  algorithm  development  rather 
than  software  optimizations. 

Software  Architecture 

To  drive  safely  in  traffic,  the  software  system  must 


be  reliable  and  robust  and  able  to  run  in  real  time. 
Boss's  software  architecture  is  fundamentally  a  dis¬ 
tributed  system,  enabling  decoupled  development 
and  testing.  In  all,  60  processes  combine  to  imple¬ 
ment  Boss's  driving  behavior.  Architecturally  these 
are  clustered  into  four  major  components:  percep¬ 
tion,  mission  planning,  behavioral  execution,  and 
motion  planning  (figure  3). 

The  perception  component  interfaces  to  Boss's 
numerous  sensors  and  fuses  the  data  into  a  coher¬ 
ent  world  model.  The  world  model  includes  the 
current  state  of  the  vehicle,  the  current  and  future 
positions  of  other  vehicles,  static  obstacles,  and  the 
location  of  the  road.  The  perception  system  uses  a 
variety  of  probabilistic  techniques  in  generating 
these  estimates. 

The  mission  planning  component  computes  the 
fastest  route  through  the  road  network  to  reach  the 
next  checkpoint  in  the  mission,  based  on  knowl¬ 
edge  of  road  blockages,  speed  limits,  and  the  nom¬ 
inal  time  required  to  make  special  maneuvers  such 
as  lane  changes  or  U-turns.  The  route  is  actually  a 
policy  generated  by  performing  gradient  descent 
on  a  value  function  calculated  over  the  road  net¬ 
work.  Thus  the  current  best  route  can  be  extracted 
at  any  time. 

The  behavioral  executive  executes  the  mission 
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plan,  accounting  for  traffic  and  other  features  in 
the  world  (Ferguson  et  al.  2008).  It  encodes  the 
rules  of  the  roads  and  how  and  when  these  rules 
should  be  violated.  It  also  implements  an  error 
recovery  system  that  is  responsible  for  handling 
off-nominal  occurrences,  such  as  crowded  inter¬ 
sections  or  disabled  vehicles.  The  executive  is 
implemented  as  a  family  of  state  machines  that 
translate  the  mission  plan  into  a  sequence  of 
motion  planning  goals  appropriate  to  the  current 
driving  context  and  conditions. 

The  motion  planning  component  takes  the 
motion  goal  from  the  behavioral  executive  and 
generates  and  executes  a  trajectory  that  will  safely 
drive  Boss  toward  this  goal,  as  described  in  the  fol¬ 
lowing  section.  Two  broad  contexts  for  motion 
planning  exist:  on-road  driving  and  unstructured 
driving. 

Software  Components 

In  this  section  we  describe  several  of  the  key  soft¬ 
ware  modules  (world  model,  moving  obstacle 
detection  and  tracking,  motion  planning,  intersec¬ 
tion  handling,  and  error  recovery)  that  enable  Boss 
to  drive  intelligently  in  traffic. 

World  Model 

Boss's  world  model  is  capable  of  representing  envi¬ 
ronments  containing  both  static  and  moving 
obstacles.  While  this  representation  was  developed 
for  the  Urban  Challenge,  it  can  be  also  used  more 
broadly  for  general  autonomous  driving.  The 
world  model  includes  the  road  network,  static 
obstacles,  tracked  vehicles,  and  the  position  and 
velocity  of  Boss.  The  road  network  defines  where 
and  how  vehicles  are  allowed  to  drive,  traffic  rules 
(such  as  speed  limits),  and  the  intersections 
between  roads.  The  road  representation  captures 
sufficient  information  to  describe  the  shape  (the 
number  of  lanes,  lane  widths,  and  geometry)  and 
the  rules  governing  driving  on  that  road  (direction 
of  travel,  lane  markings,  stop  lines,  and  so  on). 

Static  obstacles  are  those  that  do  not  move  dur¬ 
ing  some  observation  period.  This  includes  obsta¬ 
cles  off  and  on  road.  Examples  of  static  obstacles 
include  buildings,  traffic  cones,  and  parked  cars.  In 
contrast,  dynamic  obstacles  are  defined  as  objects 
that  may  move  during  an  observation  period.  With 
this  definition  all  vehicles  participating  actively  in 
traffic  are  dynamic  obstacles  even  if  they  are  tem¬ 
porarily  stopped  at  an  intersection.  By  these  defi¬ 
nitions  an  object  is  either  static  or  dynamic;  it 
should  not  be  represented  in  both  classes.  An 
object  can,  however,  change  from  static  to  dynam¬ 
ic  and  vice  versa:  a  person  may,  for  example,  get 
into  a  parked  car  and  actively  begin  participating 
in  traffic.  The  information  about  whether  an 
object  is  static  or  dynamic  is  not  tied  exclusively  to 


its  motion;  it  is  an  interpretation  of  the  current 
state  of  the  world  around  the  vehicle. 

Each  dynamic  obstacle  is  represented  by  a  series 
of  oriented  rectangles  and  velocities.  Each  rectan¬ 
gle  represents  the  state,  or  predicted  state,  of  the 
obstacle  at  an  instant  in  time  over  the  next  few  sec¬ 
onds.  Additionally,  each  obstacle  can  be  flagged  as 
either  moving  or  not  moving  and  as  either  observed 
moving  or  not  observed  moving.  An  obstacle  is 
flagged  as  moving  if  its  instantaneous  velocity  esti¬ 
mate  is  above  some  minimum  noise  threshold. 
The  obstacle  is  not  flagged  as  observed  moving 
until  the  instantaneous  velocity  remains  above  a 
noise  floor  for  some  threshold  time.  Similarly,  a 
moving  obstacle  does  not  transition  from  observed 
moving  to  not  observed  moving  until  its  instanta¬ 
neous  velocity  remains  below  threshold  for  some 
large  amount  of  time.  These  flags  provide  a  hys- 
teretic  semantic  state  that  is  used  in  the  behavioral 
executive  to  improve  behavior  while  maintaining 
the  availability  of  full  fidelity  state  information  for 
motion  planning  and  other  modules. 

Static  obstacles  are  represented  in  a  grid  of  regu¬ 
larly  spaced  cells.  The  advantages  of  this  represen¬ 
tation  are  that  it  provides  sufficient  information 
for  motion  planning  and  is  relatively  inexpensive 
to  compute  and  query.  The  disadvantages  are  that 
this  simple  representation  makes  no  attempt  to 
classify  the  type  of  obstacle  and  uses  a  relatively 
large  amount  of  memory.  To  generate  the  map, 
range  data  is  first  segmented  into  ground  and  non¬ 
ground  returns  and  then  accumulated  in  a  map 
using  an  occupancy  gridlike  algorithm.  To  main¬ 
tain  a  stable  world  representation,  dynamic  obsta¬ 
cles  that  are  flagged  as  not  observed  moving  and 
not  moving  are  painted  into  the  map  as  obstacles. 
Once  a  dynamic  obstacle  achieves  the  state  of 
observed  moving,  the  cells  in  the  map  that  it  over¬ 
laps  are  cleared.  In  the  process  of  building  the  stat¬ 
ic  obstacle  map,  an  instantaneous  obstacle  map  is 
generated.  This  map  contains  all  static  and  dynam¬ 
ic  obstacles  (that  is,  data  associated  to  dynamic 
obstacle  hypotheses  with  the  observed  moving  flag 
set  are  not  removed).  This  map  is  much  noisier 
than  the  static  obstacle  map  and  is  used  only  for 
validating  potential  moving  obstacle  tracks  within 
the  dynamic  obstacle  tracking  algorithms. 

Moving  Obstacle  Detection  and  Tracking 

The  moving  obstacle  detection  and  tracking  algo¬ 
rithms  fuse  sensor  data  to  generate  a  list  of  any 
dynamic  obstacles  in  a  scene  (Darms,  Rybski,  and 
Urmson  2008).  The  tracking  system  is  divided  into 
two  layers:  a  sensor  layer  and  a  fusion  layer  (see  fig¬ 
ure  4).  The  sensor  layer  includes  modules  that  are 
specialized  for  each  class  of  sensor  and  runs  a  mod¬ 
ule  for  each  physical  sensor  on  the  vehicle.  The 
sensor  layer  modules  perform  sensor-specific  tasks 
such  as  feature  extraction,  validation,  association. 
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Figure  4.  The  Moving  Obstacle  Tracking  System  Is  Decomposed  into  Fusion  and  Sensor  Layers. 


and  proposal  and  observation  generation.  Each  of 
these  tasks  relies  on  sensor-specific  knowledge, 
generally  encoded  as  heuristics;  these  are  the 
important  details  that  enable  the  machinery  of  the 
Kalman  filter  to  work  correctly.  For  example,  radar 
sensors  may  generate  a  specific  class  of  false  echoes 
fhat  can  be  heuristically  filtered,  but  the  filtering 
approach  would  be  irrelevant  for  a  lidar  or  even  a 
different  class  of  radar.  By  encapsulating  these  sen¬ 
sor-specific  details  within  sensor  modules  we  were 
able  to  modularize  the  algorithms  and  maintain  a 
clean,  general  implementation  in  the  fusion  layer. 

The  fusion  layer  combines  data  from  each  of  the 
sensor  modules  into  the  dynamic  obstacle  list, 
which  is  communicated  to  other  components.  The 
fusion  layer  performs  a  global  validation  of  sensor 
observafions,  selecfs  an  appropriate  tracking  mod¬ 
el,  incorporates  measurements  into  the  model  and 
predicts  future  states  of  each  obstacle.  The  global 
validation  step  uses  the  road  network  and  instan¬ 
taneous  obstacle  map  to  validate  measurements 
generated  by  the  sensor  layer.  Any  observation 
either  too  far  from  a  road,  or  wifhouf  supporting 
data  in  the  instantaneous  obstacle  map,  is  discard¬ 
ed.  For  each  remaining  observation,  an  appropriate 
tracking  model  is  selected,  and  the  state  estimate 
for  each  obstacle  is  updated.  The  future  state  of 
each  moving  obstacle  is  generated  by  either  extrap¬ 
olating  its  track  (over  short  time  intervals)  or  by 


using  the  road  network  and  driving  context  (for 
long  time  intervals). 

Motion  Planning 

Motion  planning  is  responsible  for  moving  fhe 
vehicle  safely  through  the  environment.  To  do 
this,  it  first  creates  a  path  toward  the  goal,  then 
tracks  it  by  generating  a  set  of  candidate  trajecto¬ 
ries  following  fhe  pafh  to  varying  degrees  and 
selecting  the  best  collision-free  frajecfory  from  this 
set.  During  on-road  navigation,  the  motion  plan¬ 
ner  generates  its  nominal  path  along  the  center 
line  of  the  desired  lane  (Ferguson,  Howard,  and 
Likhachev  2008a).  A  set  of  local  goals  are  selecfed 
at  a  fixed  longitudinal  distance  down  this  center 
line  path,  varying  in  lateral  offset  to  provide  sever¬ 
al  options  for  fhe  planner.  Then,  a  model-based 
frajectory  generation  algorithm  (Howard  et  al. 
2008)  is  used  to  compute  dynamically  feasible  fra- 
jectories  to  these  local  goals.  The  resulting  trajec¬ 
tories  are  evaluated  against  their  proximity  to  stat¬ 
ic  and  dynamic  obstacles,  their  distance  from  the 
center  line  path,  their  smoothness,  and  various 
other  metrics  to  determine  the  best  trajectory  for 
execution  by  the  vehicle.  This  technique  is  suffi¬ 
ciently  fast  and  produces  smooth,  high-speed  tra¬ 
jectories  for  the  vehicle,  but  it  can  fail  fo  produce 
feasible  frajecfories  when  confronfed  with  aberrant 
scenarios,  such  as  blocked  lanes  or  extremely  tight 
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turns.  In  these  cases,  the  error  recovery  system  is 
called  on  to  rescue  the  system. 

When  driving  in  unstructured  areas  the  motion 
planner's  goal  is  to  achieve  a  specific  pose  (Fergu¬ 
son,  Howard,  and  Likhachev  2008b).  To  achieve  a 
pose,  a  lattice  planner  searches  over  vehicle  posi¬ 
tion,  orientation,  and  velocity  to  generate  a 
sequence  of  feasible  maneuvers  that  are  collision 
free  with  respect  to  static  and  dynamic  obstacles. 
This  planner  is  much  more  powerful  and  flexible 
than  the  on-road  planner,  but  it  also  requires  more 
computational  resources  and  is  limited  to  speeds 
below  15  mph.  The  recovery  system  leverages  this 
flexibility  to  navigate  congested  intersections,  per¬ 
form  difficult  U-turns,  and  circumvent  or  over¬ 
come  obstacles  that  block  the  progress  of  the  vehi¬ 
cle.  In  these  situations,  the  lattice  planner  is 
typically  biased  to  avoid  unsafe  areas,  such  as 
oncoming  traffic  lanes,  but  this  bias  can  be 
removed  as  the  situation  requires. 

Intersection  Handling 

Correctly  handling  precedence  at  intersections  is 
one  function  of  the  behavioral  executive  (Baker 
and  Dolan  2008).  Within  the  executive,  the  prece¬ 
dence  estimator  has  primary  responsibility  for 
ensuring  that  Boss  is  able  to  safely  and  robustly 
handle  intersections.  The  precedence  estimator 
encapsulates  all  of  the  algorithms  for  determining 
when  it  is  safe  to  proceed  through  an  intersection. 
Before  entering  the  intersection,  the  intersection 
must  be  clear  (both  instantaneously  and  through¬ 
out  the  time  it  predicts  Boss  will  take  to  travel 
through  the  intersection)  and  Boss  must  have 
precedence.  The  precedence  order  is  determined 
using  typical  driving  rules  (that  is,  the  first  to  arrive 
has  precedence  through  the  intersection  and  in 
cases  of  simultaneous  arrival,  yield  to  the  right). 

The  precedence  estimator  uses  knowledge  of  the 
vehicle's  route,  to  determine  which  intersection 
should  be  monitored,  and  the  world  model,  which 
includes  dynamic  obstacles  around  the  robot,  to 
calculate  the  precedence  ordering  at  the  upcoming 
intersection.  To  ensure  robustness  to  temporary 
noise  in  the  dynamic  obstacle  list,  the  algorithm 
for  computing  precedence  does  not  make  use  of 
any  information  beyond  the  geometric  properties 
of  the  intersection  and  the  instantaneous  position 
of  each  dynamic  obstacle.  Instead  of  assigning 
precedence  to  specific  vehicles,  precedence  is 
assigned  to  individual  stop  lines  at  an  intersection, 
using  a  geometric  notion  of  "occupancy"  to  deter¬ 
mine  arrival  times.  Polygons  are  computed  around 
each  stop  line,  extending  backward  along  the 
incoming  lane  by  a  configurable  distance  and 
exceeding  the  lane  geometry  by  a  configurable 
margin.  If  the  front  bumper  of  a  moving  obstacle 
is  inside  this  polygon,  it  is  considered  an  occupant 
of  the  associated  stop  line.  Boss's  front  bumper  is 


added  to  the  pool  of  front  bumpers  and  is  treated 
in  the  same  manner  to  ensure  equitable  estimation 
of  arrival  times.  Figure  5  shows  a  typical  occupan¬ 
cy  polygon  extending  three  meters  back  along  the 
lane  from  the  stop  line.  Once  the  polygon  becomes 
occupied  the  time  is  recorded  and  used  to  estimate 
precedence.  To  improve  robustness  of  the  dynam¬ 
ic  obstacle  list,  the  occupancy  state  is  not  cleared 
instantly  when  there  is  no  obstacle  in  the  polygon. 
Instead,  the  polygon  remains  occupied  for  a  small 
amount  of  time.  Should  an  obstacle  reappear  in 
the  polygon  in  this  time,  the  occupancy  state  of 
the  polygon  is  maintained.  This  allows  moving 
obstacles  to  flicker  out  of  existence  for  short  inter¬ 
vals  without  disturbing  the  precedence  ordering. 

In  cases  where  Boss  stops  at  an  intersection  and 
none  of  the  other  vehicles  move.  Boss  will  wait  10 
seconds  before  breaking  the  deadlock.  In  this  case 
the  precedence  estimator  assumes  that  the  other 
traffic  at  the  intersection  is  not  taking  precedence 
and  asserts  that  Boss  has  precedence.  In  these  situ¬ 
ations  other  elements  of  the  behavioral  executive 
limit  the  robot's  maximum  speed  to  5  mph,  caus¬ 
ing  it  to  proceed  cautiously  through  the  intersec¬ 
tion.  Once  the  robot  has  begun  moving  through 
the  intersection,  there  is  no  facility  for  aborting  the 
maneuver,  and  the  system  instead  relies  on  the 
motion  planner's  collision-avoidance  algorithms 
and,  ultimately,  the  intersection  recovery  algo¬ 
rithms  to  guarantee  safe,  forward  progress  should 
another  robot  enter  the  intersection  at  the  same 
time. 

Error  Recovery 

Despite  best  efforts  to  implement  a  correct  and  reli¬ 
able  system,  there  will  always  be  bugs  in  the  imple¬ 
mentation  and  unforeseen  and  possibly  illegal 
behavior  by  other  vehicles  in  the  environment. 
The  error  recovery  system  was  designed  to  ensure 
that  Boss  handles  these  situations  and  "never  gives 
up."  To  this  end,  a  good  recovery  algorithm  should 
generate  a  sequence  of  nonrepeating  motion  goals 
that  gradually  accept  greater  risk;  it  should  adapt 
behavior  to  the  driving  context,  since  each  context 
includes  road  rules  that  should  be  adhered  to  if 
possible  and  disregarded  only  if  necessary  in  a 
recovery  situation;  and  finally,  the  recovery  system 
should  also  be  kept  as  simple  as  possible  to  reduce 
the  likelihood  of  introducing  further  software 
bugs. 

Our  approach  is  to  use  a  small  family  of  error 
recovery  modes  and  maintain  a  recovery  level. 
Each  recovery  mode  corresponds  to  a  nominal 
driving  context  (driving  down  a  lane,  handling  an 
intersection  or  maneuvering  in  an  unstructured 
area).  When  in  normal  operation,  the  recovery  lev¬ 
el  is  zero,  but  as  failures  occur,  it  is  incremented. 
The  recovery  level  is  used  to  encode  the  level  of 
risk  that  is  acceptable.  It  is  important  to  note  that 
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Figure  5.  A  Typical  Occupancy  Polygon  for  a  Stopline. 


the  recovery  goal-selection  process  does  not  explic¬ 
itly  incorporate  any  surrounding  obstacle  data, 
making  it  intrinsically  robust  to  transient  environ¬ 
mental  effects  or  perception  artifacts  that  may 
cause  spurious  failures  in  other  subsystems. 

All  recovery  maneuvers  are  specified  as  a  set  of 
poses  to  be  achieved  and  utilize  the  movement 
zone  lattice  planner  to  generate  paths  through 
potentially  complicated  situations.  The  successful 
completion  of  any  recovery  goal  returns  the  sys¬ 
tem  to  normal  operation,  resetting  the  recovery 
level.  Should  the  same,  or  a  very  similar,  normal 
goal  fail  immediately  after  a  seemingly  successful 
recovery  sequence,  the  previous  recovery  level  is 
reinstated  instead  of  simply  incrementing  from 
zero  to  one.  This  bypasses  the  recovery  levels  that 
presumably  failed  to  get  the  system  out  of  trouble 
and  immediately  selects  a  goal  at  a  higher  level. 

The  on-road  recovery  algorithm  involves  gener¬ 
ating  goals  at  incrementally  greater  distances  down 
the  current  road  lane  to  some  maximum  distance. 
These  forward  goals  (goals  1,  2,  3  in  figure  6a)  are 
constrained  to  remain  in  the  original  lane  of  trav¬ 
el.  If  none  of  the  forward  goals  succeed,  a  goal  is 
selected  a  short  distance  behind  the  vehicle  (goal 
4)  with  the  intent  of  backing  up  to  provide  a  new 
perspective.  After  backing  up,  the  sequence  of  for¬ 
ward  goals  is  allowed  to  repeat  once  more  with  an 
offset,  after  which  continued  failure  triggers  high¬ 


er-risk  maneuvers.  For  example,  if  there  is  a  lane 
available  in  the  opposing  direction,  the  road  is 
eventually  marked  as  blocked,  and  the  system 
issues  a  U-turn  goal  (goal  5)  and  attempts  to  follow 
an  alternate  path  to  the  goal.  The  intersection 
recovery  algorithm  involves  attempting  to  navi¬ 
gate  alternate  paths  through  an  intersection  and 
then  alternative  routes  through  the  road  network. 
The  desired  path  through  an  intersection  is  a 
smooth  interpolation  between  the  incoming  and 
outgoing  lanes  that  may  be  blocked  by  some  obsta¬ 
cle  in  the  intersection  or  may  be  infeasible  to  drive. 
Thus,  the  first  recovery  level  (goal  1  in  figure  6b)  is 
to  try  to  achieve  a  pose  slightly  beyond  the  origi¬ 
nal  goal  using  the  lattice  planner,  with  the  whole 
intersection  as  its  workspace.  This  quickly  recovers 
the  system  from  small  or  spurious  failures  in  inter¬ 
sections  and  compensates  for  intersections  with 
turns  that  are  tighter  than  the  vehicle  can  easily 
make.  If  the  first  recovery  goal  fails,  alternate 
routes  out  of  the  intersection  (goals  2,  3,  4)  are 
attempted  until  all  alternate  routes  to  the  goal  are 
exhausted.  Thereafter,  the  system  removes  the  con¬ 
straint  of  staying  in  the  intersection  and  selects 
goals  incrementally  further  out  from  the  intersec¬ 
tion  (goals  5,  6). 

The  unstructured  zone  recovery  procedure  varies 
depending  on  the  current  objective.  Initially,  the 
algorithm  selects  a  sequence  of  goals  in  a  regular. 
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Figure  6.  Boss  Uses  Context-Specific  Recovery  Goals  to  Provide  Robustness 
to  Software  Bugs  and  Abnormal  Behavior  of  Other  Vehicles. 


triangular  pattern,  centered  on  the  original  goal  as 
shown  in  figure  6(c).  If,  after  attempting  this  com¬ 
plete  pattern,  the  vehicle  continues  to  fail  to  make 
progress,  a  more  specialized  recovery  process  is 
invoked.  For  parking  spot  goals,  the  pattern  is  con¬ 
tinuously  repeated  with  small  incremental  angular 
offsets,  based  on  the  assumption  that  the  parking 
spot  will  eventually  be  correctly  perceived  as  emp¬ 
ty,  or  the  obstacle  (temporarily)  in  the  spot  will 
move.  If  the  current  goal  is  to  exit  the  parking  lot, 


the  algorithm  tries  first  to  use  alternative  exits,  if 
fhose  exits  fail,  then  the  algorithm  issues  goals  to 
locations  on  the  road  network  outside  of  the  park¬ 
ing  lot,  giving  the  lattice  planner  free  reign  in  gen¬ 
erating  a  solution  out  of  fhe  parking  lof. 

An  independent  last-ditch  recovery  system 
monitors  the  vehicle's  motion.  If  the  vehicle  does 
not  move  at  least  one  meter  every  five  minutes,  it 
overrides  any  other  goals  with  a  randomly  selected 
local  motion.  The  hope  is  that  some  small  local 
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Figure  7.  The  Urban  Challenge  Final  Event  Was  Held  on  a  Road  Network  in  and  around  the  Former  George  Air  Force  Base. 


motion  will  either  physically  dislodge  the  vehicle 
or,  hy  changing  the  point  of  view,  clear  any  arti¬ 
facts  due  to  noisy  perceptions  that  are  causing  the 
failure.  Upon  successful  completion  of  this 
motion,  the  recovery  level  is  reset,  enabling  the 
vehicle  to  continue  on  its  way.  This  functionality 
is  analogous  to  the  "wander"  behavior  (Brooks 
1986). 

The  components  described  in  this  section  are 
just  a  few  examples  of  the  autonomy  components 
that  caused  Boss  to  drive  safely.  For  a  more  in 
depth  overview,  we  point  the  reader  to  Urmson  et 
al.  (2008) 

Performance 

During  testing  and  development,  Boss  completed 
more  than  3000  kilometers  of  autonomous  driv¬ 
ing.  The  most  visible  portion  of  this  testing 


occurred  during  qualification  and  competition  at 
the  DARPA  Urban  Challenge.  The  event  was  held 
at  the  former  George  Air  Force  Base  in  Victorville, 
California,  USA  (figure  7).  During  qualification, 
vehicles  were  required  to  demonstrate  their  abili¬ 
ties  in  three  test  areas.  Area  A  required  the 
autonomous  vehicles  to  merge  into  and  turn 
across  dense  moving  traffic.  Vehicles  had  to  judge 
the  size  of  gaps  between  moving  vehicles,  assess 
safety,  and  then  maneuver  without  excessive 
delay.  For  many  of  the  vehicles,  this  was  the  most 
difficult  challenge,  as  it  involved  significant  rea¬ 
soning  about  moving  obstacles.  Area  B  was  a  rela¬ 
tively  long  road  course  that  wound  through  a 
neighborhood.  Vehicles  were  challenged  to  find 
their  way  through  the  road  network  while  avoid¬ 
ing  parked  cars,  construction  areas,  and  other  road 
obstacles  but  did  not  encounter  moving  traffic. 
Area  C  was  a  relatively  short  course  but  required 
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autonomous  vehicles  to  demonstrate  correct 
behavior  with  traffic  at  four-way  intersections  and 
to  demonstrate  rerouting  around  an  unexpectedly 
blocked  road.  Boss  demonstrated  safe  and  reliable 
driving  across  all  of  these  areas  and  was  the  first 
vehicle  to  qualify  for  the  Urban  Challenge  compe¬ 
tition. 

During  the  final  event,  the  11  finalists  were 
required  to  drive  52  miles  in  and  around  the  base 
while  interacting  with  each  other  and  fifty  human- 
driven  vehicles.  Six  of  the  11  teams  that  entered 
the  final  competition  completed  the  course 
(Buehler,  Lagnemma,  and  Singh  2008).  Boss  fin¬ 
ished  approximately  19  minutes  faster  than  the 
second-place  vehicle,  Junior  (from  Stanford  Uni¬ 
versity),  and  26  minutes  ahead  of  the  third-place 
vehicle,  Odin  (from  Virginia  Polytechnic  Institue 
and  State  University).  The  vehicles  from  Cornell 
University,  the  Massachusetts  Institute  of  Tech¬ 
nology,  and  the  University  of  Pennsylvania  round¬ 
ed  out  the  finishers,  but  each  of  these  vehicles 
required  some  human  intervention  to  complete 
the  course.  All  of  the  vehicles  that  completed  the 
challenge  performed  impressively  with  only  rela¬ 
tively  minor  driving  errors:  Boss  twice  incorrectly 
determined  that  it  needed  to  make  a  U-turn,  result¬ 
ing  in  driving  an  unnecessary  2  miles;  Junior  had 
a  minor  bug  that  caused  it  to  repeatedly  loop  twice 
through  one  section  of  the  course;  and  Odin 
incurred  a  significant  GPS  error  that  caused  it  to 
drive  partially  off  the  road  for  small  periods  of 
time.  Despite  these  glitches,  these  vehicles  repre¬ 
sent  a  new  state  of  the  art  for  autonomous  urban 
driving. 

During  the  final  competition,  the  value  of  Boss's 
error  recovery  system  was  demonstrated  on  17 
occasions.  All  but  3  of  these  errors  were  resolved 
by  the  first  error  recovery  level,  which,  in  general, 
meant  that  an  outside  observer  would  not  detect 
that  anything  untoward  had  occurred.  The 
remaining  three  recoveries  consumed  more  than 
half  of  the  11  minutes  the  vehicle  spent  in  error 
recovery  modes  and  were  generally  due  to  soft¬ 
ware  bugs  or  sensor  artifacts.  For  example,  the  U- 
turn  mistakes  described  above  were  due  to  a  com¬ 
bination  of  perception  and  planning  bugs  that 
caused  the  system  to  believe  that  the  path  forward 
along  the  current  lane  was  completely  and  per¬ 
sistently  blocked  by  other  traffic  when  there  was, 
in  fact,  no  traffic  at  all.  After  repeated  attempts  to 
move  forward,  the  recovery  system  declared  the 
lane  fully  blocked,  commanded  a  U-turn,  and  fol¬ 
lowed  an  alternate  route  to  its  goal.  While  subop- 
timal,  this  behavior  demonstrated  the  recovery 
system's  ability  to  compensate  for  subtle  but 
potentially  lethal  bugs  in  the  software  system.  The 
end  result  is  that  Boss  demonstrated  robust  auton¬ 
omy  in  the  face  of  the  unexpected  and  its  own 
shortcomings. 


Discussion 

The  Urban  Challenge  was  a  compelling  demon¬ 
stration  of  the  state  of  the  art  in  autonomous 
robotic  vehicles.  It  has  helped  to  reinvigorate  belief 
in  the  promise  of  autonomous  vehicles.  While  the 
Urban  Challenge  is  not  wholly  representative  of 
daily  driving,  it  does  represent  a  significant  step 
toward  achieving  general  driving.  Some  of  the 
remaining  challenges  include  diagnostics,  valida¬ 
tion,  improved  modeling,  and  autonomous  vehi¬ 
cle  and  driver  interaction. 

Diagnostics:  It  is  essential  that  autonomous  vehi¬ 
cles  understand  when  they  are  in  a  safe  working 
condition  and  when  they  should  not  be  operated. 
Reliable  detection  and  diagnosis  of  certain  classes 
of  sensor  and  operational  faults  are  very  difficult. 

Validation:  Prior  to  deploying  something  as  safe¬ 
ty  critical  as  an  autonomous  vehicle,  we  must  have 
confidence  that  the  vehicle  will  behave  as  desired. 
While  there  has  been  significant  progress  in  both 
systems  engineering  and  automated  software  vali¬ 
dation,  neither  field  is  yet  up  to  the  task  of  vali¬ 
dating  these  complex  autonomous  systems. 

Improved  modeling:  The  Urban  Challenge  offered 
only  a  subset  of  the  interactions  a  driver  faces  dai¬ 
ly;  there  were  no  pedestrians,  no  traffic  lights,  no 
adversarial  drivers,  and  no  bumper-to-bumper  traf¬ 
fic.  A  sufficient  and  correct  model  of  the  world  for 
complex  urban  driving  tasks  is  yet  to  be  developed. 

Autonomous  vehicle  and  driver  interaction:  By  not 
allowing  drivers  in  the  Urban  Challenge  robots, 
this  problem  was  irrelevant,  but  when 
autonomous  vehicles  are  deployed,  the  sharing  of 
control  between  driver  and  vehicle  will  be  critical¬ 
ly  important.  We  are  only  just  beginning  to  under¬ 
stand  how  to  approach  this  problem. 

Many  of  these  items  will  best  be  addressed  by 
applying  artificial  intelligence  and  machine-learn¬ 
ing  techniques  since  they  require  complicated 
models  that  are  difficult  to  explicitly  encode. 
Despite  these  challenges,  autonomous  vehicles  are 
here  to  stay.  We  have  already  begun  work  with 
Caterpillar  to  use  techniques  and  algorithms  devel¬ 
oped  during  the  Urban  Challenge  to  automate 
large  off-highway  trucks  for  mining  applications. 
The  automotive  industry  is  interested  in  improv¬ 
ing  existing  driver  assistance  systems  by  using 
autonomous  vehicle  techniques  to  further  reduce 
traffic  accidents  and  fatalities  (global  traffic  acci¬ 
dent  fatalities  exceed  1.2  million  people  each  year), 
reduce  congestion  on  crowded  roads,  and  improve 
fuel  economy.  The  question  is  no  longer  if  vehicles 
will  become  autonomous  but  when  and  where  can  I 
get  one? 
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